The invention relates to a method and a device for displaying an image of the surroundings, in particular the image of the surroundings of a vehicle. In addition, the invention relates to a surround view system for a vehicle combination.
Vehicles are increasingly being equipped with rear view systems or surround view systems which calculate a display image from camera images, which display image supports the driver of a vehicle during the performance of driving maneuvers such as, for example, during parking or reversing. The view of a vehicle driver is, as a rule, restricted especially during reversing, so that thanks to the support of the display system he is able to take decisions in order to maneuver his vehicle safely. As a result of the display of processed camera images, the driver can, in this case, be provided with an image of the vehicle surroundings, which is as realistic as possible. As a result, it is possible to support the driver efficiently during his driving maneuvers.
On a vehicle combination, in which a towing vehicle is coupled to a trailer, a camera directed backwards on the towing vehicle generally senses, to a large extent, the trailer. In this case, the trailer obscures the view of the further surrounding area at the rear. A camera system, in which cameras are exclusively arranged on the towing vehicle, cannot therefore allow a sufficient rear view for such a vehicle combination. Therefore, an additional camera is occasionally arranged on the trailer. If the images of this camera on the trailer are displayed to a driver/operator in the towing vehicle, this may possibly lead to confusion since the alignment of the camera on the trailer does not correspond to the alignment of the towing vehicle.
It is therefore an object of the invention to provide a method and a device for displaying an image of the surroundings of a vehicle combination, in which the image of the surroundings around the entire vehicle combination can be shown as completely and realistically as possible, so that the driver is efficiently supported in his driving decisions and orientation.
This object is achieved according to the invention by a method having the features indicated in claim 1.
The invention therefore provides a method for displaying an image of the surroundings of a vehicle combination, having the steps of providing first image data sensed by sensors and providing second image data sensed by sensors. The first image data is, in this case, sensed and provided by means of a vehicle sensor unit arranged on a first vehicle. The second image data is sensed and provided by means of a second vehicle sensor unit arranged on a second vehicle. The first vehicle and the second vehicle can, in this case, be mechanically coupled to one another and consequently form a vehicle combination. The vehicle further includes a step to establish a three-dimensional position of the second vehicle in relation to the first vehicle; and combining the first image data and the second image data using the established three-dimensional position of the second vehicle in relation to the first vehicle. As a result of combining the first image data and the second image data, an image of the surroundings of the vehicle combination is obtained in this case. The obtained image of the surroundings is then displayed on a vehicle display.
The invention further provides a device for displaying an image of the surroundings of a vehicle combination having the features indicated in claim 11.
The invention therefore provides a device for displaying an image of the surroundings of a vehicle combination having a first vehicle sensor unit, a second vehicle sensor unit and a calculation unit. The first vehicle sensor unit is arranged on a first vehicle and is designed to provide first image data. The second vehicle sensor unit is arranged on a second vehicle and is designed to provide two image data. The first vehicle and the second vehicle can be mechanically coupled to one another and thus form a vehicle combination. The calculation unit is designed to establish a three-dimensional position of the second vehicle in relation to the first vehicle. The calculation unit is further designed to combine first image data and second image data using the established three-dimensional position of the second vehicle in relation to the first vehicle. As a result, an image of the surroundings of the vehicle combination is obtained. The obtained image of the surroundings can be displayed on a vehicle display connected to the calculation unit.
The idea which forms the basis of the invention is to obtain a common image of the surroundings for a combination of two vehicles that are coupled to one another. To this end, image data originating both from image sensors, which are arranged in a clamped manner on a towing vehicle of the vehicle, and image data originating from image sensors, which are arranged on the trailer of the vehicle combination, can be combined with one another. All of this image data is, in this case, transferred into a common reference system. This common reference system is preferably a reference system based on the towing vehicle. In this way, a driver/operator in the towing vehicle can perceive a thus represented image of the surroundings particularly easily and can subsequently make decisions efficiently regarding driving maneuvers.
As a result of the combination of image data originating from image sensors of both the towing vehicle and the trailer, a particularly large area of the surroundings around the entire vehicle combination can, in this case, be sensed and shown. In particular, those image areas which are obscured by a trailer can consequently also be shown to a driver/operator in a towing vehicle. As a result, a safe maneuvering of the vehicle combination is made possible, in particular during reversing.
The evaluation of the three-dimensional position between the two vehicles of such a vehicle combination during the combination of the image data makes it possible, in this case, to combine all of the image data particularly well and particularly reliably. In particular, the image data of a camera system arranged on a trailer can, as a result, be translated very well into the reference system of the towing vehicle.
According to one embodiment, image areas of the first vehicle sensor unit and of the second vehicle sensor unit overlap at least partially. Due to the at least partial overlapping of the image areas of the vehicle sensor units, it is possible to achieve an imaging of the vehicle surroundings, which is as complete as possible. In addition, overlapping image areas of individual vehicle sensor units also make it possible, thanks to the analysis of the overlapping areas, to combine the image data so as to obtain a particularly good common image of the surroundings.
According to one embodiment, the step to establish the three-dimensional position of the second vehicle in relation to the first vehicle detects the second vehicle in the first image data provided. Based on the detection result for the detection of the second vehicle in the first image data, the three-dimensional position of the second vehicle can subsequently be established in relation to the first vehicle. In this way, it is possible to use the image data of the first vehicle sensor unit in order to establish the position of the second vehicle in relation to the first vehicle. Consequently, no additional separate sensors, which detect the three-dimensional position of the two vehicles relative to one another, are required.
According to another embodiment, establishing the three-dimensional position of the second vehicle in relation to the first vehicle includes establishing an angle and/or a distance between the first vehicle and the second vehicle. The established angle between the first vehicle and the second vehicle can, for example, be the angular position of a trailer drawbar. As a result of establishing the angular position between the two vehicles and the distance between the two vehicles, the three-dimensional position of the second vehicle and consequently also the position of the second vehicle sensor unit in relation to the first vehicle or respectively the first vehicle sensor unit can be clearly determined.
According to another embodiment, the method for displaying an image of the surroundings of a vehicle combination includes a step to obtain a graphical representation of the second vehicle. The obtained graphical representation of the second vehicle can subsequently be integrated into the obtained image of the surroundings. For example, the obtained graphical representation of the second vehicle can be semi-transparently inserted into the obtained image of the surroundings. Likewise, it is possible to show, for example, the outer contours of the second vehicle in the obtained image of the surroundings as lines or similar. In this way, a driver/operator additionally receives information regarding the position of the second vehicle in the surroundings, in the image of the surroundings shown.
According to one embodiment, a three-dimensional model of the second vehicle is obtained in the step to obtain a graphical representation of the second vehicle. First image data of the second vehicle can then be projected onto outer surfaces of the obtained three-dimensional model of the second vehicle. Consequently, a particularly realistic representation of the second vehicle in the image of the surroundings is possible.
According to another embodiment, the method for displaying an image of the surroundings of a vehicle combination includes a step to transform the first image data sensed by sensors and the second image data sensed by sensors into a predetermined projection surface. The step to combine the first image data and the second image data can subsequently obtain the image of the surroundings using the transformed first image data and the transformed second image data. The transformation into a predetermined projection surface can, for example, involve a projection of the image data into a bird's eye view, a transformation of the image data into the plane of the road on which the first vehicle and the second vehicle are located, a projection of the image data onto the outer surfaces of the second vehicle or any other projection surface/plane. Thanks to the transformation of the image data into a predefined projection surface prior to the combination, it is subsequently possible to combine the image data particularly easily.
According to another embodiment, the method for displaying an image of the surroundings of a vehicle combination includes a step to transform the obtained image of the surroundings into a predetermined perspective. Thanks to the transformation of the obtained image of the surroundings into a predetermined perspective, for example into a bird's eye perspective or any other projection plane, the combined image data can be shown to a driver/operator in a view which makes it possible to perceive the image data particularly efficiently.
According to another embodiment, the first image data and the second image data are in each case supplied by at least one camera of the first vehicle sensor unit or respectively of the second vehicle sensor unit. Alternatively, vehicle sensor units having more than one camera are also possible.
According to another embodiment, the method for displaying an image of the surroundings includes a step to calculate extrinsic parameters of the second vehicle sensor unit. In particular, this step can include calculating extrinsic camera parameters of a camera of the second vehicle sensor unit. The extrinsic camera parameters describe the position and orientation of the second vehicle sensor unit in the space, in particular the position and orientation of the second vehicle sensor unit in relation to the first vehicle or respectively the first vehicle sensor unit. The extrinsic camera parameters can, in this case, include a translation of the second vehicle sensor unit in relation to the first vehicle sensor unit in the x-, y- and z-directions as well as rotation about the x-axis, the y-axis and the z-axis.
According to another embodiment, the extrinsic parameters of the second vehicle sensor unit are calculated in real time. In particular, the extrinsic parameters can be calculated separately for each image (frame) a camera of the second vehicle sensor unit.
According to another embodiment, the device for displaying an image of the surroundings of a vehicle combination includes a graphics processing unit (GPU). Graphics processing units are particularly well suited to processing image or respectively graphic data efficiently. In this way, a distortion correction of the image data provided by the first vehicle sensor unit or respectively the second vehicle sensor unit, a transformation of the image data into a desired perspective, the determination of the extrinsic camera parameters as well as the particularly efficient combination of the first image data and the second image data by a corresponding graphics processing unit are made possible. In particular, the processing of the corresponding data in real time is made possible by the use of a graphics processing unit.
According to another embodiment, the device for displaying an image of the surroundings further includes a position sensor which is designed to sense an angle and/or a distance between the first vehicle and the second vehicle. The angle or respectively distance thus sensed can then be provided to the calculation unit. Using an external or respectively separate position sensor unit makes it possible to evaluate the angle or respectively distance between the first vehicle and second vehicle simply and precisely.
According to another embodiment, the first vehicle sensor unit and/or the second vehicle sensor unit respectively include(s) at least one vehicle camera.
The invention further provides a surround view system for a vehicle combination having a first vehicle and a second vehicle mechanically coupled to the first vehicle. The surround view system includes a device for displaying an image of the surroundings according to the invention and a vehicle display which is arranged in the first vehicle and which is designed to display the obtained image of the surroundings.
Further embodiments and advantages of the invention are set out by the following description, with reference to the attached drawings, wherein: